The Future of Urban Mobility AI: Can Foundation Models Outperform Classic ML & DL Models For Crowd Prediction?¶

Original Paper: Zhang et al. (2017) "Deep Spatio-Temporal Residual Networks for Citywide Crowd Flows Prediction" (Microsoft Research)¶

Zhang, J., Zheng, Y., & Qi, D. (2017). Deep Spatio-Temporal Residual Networks for Citywide Crowd Flows Prediction. Proceedings of the AAAI Conference on Artificial Intelligence, 31(1). https://doi.org/10.1609/aaai.v31i1.10735

Home page: https://www.microsoft.com/en-us/research/publication/deep-spatio-temporal-residual-networks-for-citywide-crowd-flows-prediction/

Preprint: https://arxiv.org/pdf/1610.00081

Goal¶

The goal is to predict one step ahead

image-4.png

Datasets¶

image.png

  • TaxiBJ: the last four weeks as the testing data, and all data before that as training data.
  • BikeNYC: the last 10 days are chosen as testing data, and the others as training data.

Results¶

image-3.png image-2.png

Code¶

The original repo (referenced on the homepage) does not exist anymore: https://github.com/lucktroy/DeepST/tree/master/scripts/papers/AAAI17

Therefore, I forked: https://github.com/topazape/ST-ResNet as a starting point.

BikeNYC Experiment¶

In [1]:
# [dataset]
data_files = ["./datasets/BikeNYC/NYC14_M16x8_T60_NewEnd.h5"]
holiday_file = ""
meteorol_file = ""
T = 24  # time step of one hour
len_closeness = 3
len_period = 1
len_trend = 1
period_interval = 1
trend_interval = 7
len_test = 672
use_meta = True

# [model]
nb_flow = 2
map_height = 8
map_width = 16
nb_residual_unit = 4

# [learning]
epochs = 50
batch_size = 32
learning_rate = 0.0002
In [2]:
import tables
import numpy as np
In [3]:
def remove_incomplete_days(dat: tables.file.File, T: int) -> tuple[np.ndarray, np.ndarray]:
    # 20140425 has 24 timestamps, which does not appear in `incomplete_days` in the original implementation.
    # So I reimplemented it in a different way.
    data = dat.root.data.read()
    timestamps = dat.root.date.read().astype(str)

    dates, values = np.vstack(
        np.frompyfunc(lambda x: (x[:8], x[8:]), 1, 2)(timestamps)
    )
    # label encoding
    uniq_dates, labels = np.unique(dates, return_inverse=True)
    # groupby("labels")["values"].sum() != sum(range(1, 49))
    incomplete_days = uniq_dates[
        np.where(np.bincount(labels, values.astype(int)) != sum(range(1, (T + 1))))[
            0
        ]
    ]
    del_idx = np.where(np.isin(dates, incomplete_days))[0]
    new_data = np.delete(data, del_idx, axis=0)
    new_timestamps = np.delete(timestamps, del_idx)
    return new_data, new_timestamps
In [4]:
dat=tables.open_file(data_files[0],mode="r")
data=dat.root.data.read()
data
Out[4]:
array([[[[ 0.,  0.,  1., ...,  0.,  0.,  0.],
         [ 0.,  1.,  1., ...,  1.,  0.,  0.],
         [ 0.,  0.,  2., ...,  2.,  2.,  0.],
         ...,
         [ 0.,  0.,  0., ...,  3.,  1.,  0.],
         [ 0.,  0.,  0., ...,  0.,  0.,  0.],
         [ 0.,  0.,  0., ...,  0.,  0.,  1.]],

        [[ 0.,  0.,  0., ...,  0.,  0.,  0.],
         [ 0.,  0.,  0., ...,  1.,  0.,  0.],
         [ 0.,  0.,  0., ...,  1.,  0.,  0.],
         ...,
         [ 0.,  0.,  0., ...,  0.,  1.,  0.],
         [ 0.,  0.,  0., ...,  2.,  0.,  0.],
         [ 0.,  0.,  0., ...,  0.,  0.,  0.]]],


       [[[ 0.,  0.,  0., ...,  0.,  0.,  0.],
         [ 0.,  0.,  0., ...,  0.,  0.,  0.],
         [ 0.,  0.,  0., ...,  0.,  0.,  0.],
         ...,
         [ 0.,  0.,  0., ...,  0.,  1.,  2.],
         [ 0.,  0.,  0., ...,  0.,  0.,  0.],
         [ 0.,  0.,  0., ...,  1.,  0.,  1.]],

        [[ 0.,  0.,  0., ...,  0.,  0.,  0.],
         [ 0.,  0.,  0., ...,  0.,  0.,  0.],
         [ 0.,  0.,  1., ...,  0.,  0.,  0.],
         ...,
         [ 0.,  0.,  0., ...,  0.,  0.,  0.],
         [ 0.,  0.,  0., ...,  1.,  1.,  0.],
         [ 0.,  0.,  0., ...,  0.,  0.,  1.]]],


       [[[ 0.,  0.,  0., ...,  0.,  0.,  0.],
         [ 0.,  0.,  0., ...,  1.,  0.,  0.],
         [ 0.,  0.,  0., ...,  1.,  0.,  0.],
         ...,
         [ 0.,  0.,  0., ...,  0.,  2.,  0.],
         [ 0.,  0.,  0., ...,  0.,  0.,  0.],
         [ 0.,  0.,  0., ...,  0.,  0.,  0.]],

        [[ 0.,  0.,  0., ...,  0.,  0.,  0.],
         [ 0.,  0.,  0., ...,  1.,  0.,  0.],
         [ 0.,  0.,  0., ...,  2.,  1.,  0.],
         ...,
         [ 0.,  0.,  0., ...,  1.,  0.,  0.],
         [ 0.,  0.,  0., ...,  0.,  0.,  0.],
         [ 0.,  0.,  0., ...,  0.,  0.,  0.]]],


       ...,


       [[[ 0.,  0.,  5., ...,  0.,  0.,  0.],
         [ 0.,  2.,  6., ...,  6.,  0.,  0.],
         [ 0.,  5., 22., ..., 14.,  8.,  0.],
         ...,
         [ 0.,  0.,  4., ...,  9.,  5.,  2.],
         [ 0.,  0.,  3., ...,  7.,  3.,  1.],
         [ 0.,  0.,  0., ...,  4.,  6.,  8.]],

        [[ 0.,  0.,  2., ...,  0.,  0.,  0.],
         [ 0.,  2.,  2., ...,  9.,  0.,  0.],
         [ 0.,  0., 10., ..., 13.,  7.,  0.],
         ...,
         [ 0.,  0.,  3., ...,  7.,  4.,  1.],
         [ 0.,  0.,  2., ...,  3.,  2.,  2.],
         [ 0.,  0.,  0., ...,  2.,  1.,  1.]]],


       [[[ 0.,  0.,  5., ...,  0.,  0.,  0.],
         [ 0.,  1.,  4., ...,  5.,  0.,  0.],
         [ 0.,  1.,  7., ...,  3.,  7.,  0.],
         ...,
         [ 0.,  0.,  3., ...,  8.,  1.,  0.],
         [ 0.,  0.,  2., ...,  4.,  0.,  1.],
         [ 0.,  0.,  0., ...,  2.,  4.,  3.]],

        [[ 0.,  0.,  2., ...,  0.,  0.,  0.],
         [ 0.,  1.,  5., ...,  7.,  0.,  0.],
         [ 0.,  0.,  8., ...,  3.,  5.,  0.],
         ...,
         [ 0.,  0.,  0., ...,  2.,  1.,  1.],
         [ 0.,  0.,  2., ...,  0.,  1.,  0.],
         [ 0.,  0.,  0., ...,  0.,  0.,  1.]]],


       [[[ 0.,  0.,  1., ...,  0.,  0.,  0.],
         [ 0.,  0.,  3., ...,  4.,  0.,  0.],
         [ 0.,  2.,  2., ...,  4.,  6.,  0.],
         ...,
         [ 0.,  0.,  0., ..., 12.,  0.,  2.],
         [ 0.,  0.,  1., ...,  3.,  1.,  0.],
         [ 0.,  0.,  0., ...,  2.,  2.,  2.]],

        [[ 0.,  0.,  0., ...,  0.,  0.,  0.],
         [ 0.,  0.,  2., ...,  3.,  0.,  0.],
         [ 0.,  0.,  5., ...,  0.,  2.,  0.],
         ...,
         [ 0.,  0.,  0., ...,  1.,  0.,  0.],
         [ 0.,  0.,  1., ...,  1.,  0.,  0.],
         [ 0.,  0.,  0., ...,  2.,  0.,  2.]]]], shape=(4392, 2, 16, 8))
In [5]:
datasets = remove_incomplete_days(dat,T)
datasets
Out[5]:
(array([[[[ 0.,  0.,  1., ...,  0.,  0.,  0.],
          [ 0.,  1.,  1., ...,  1.,  0.,  0.],
          [ 0.,  0.,  2., ...,  2.,  2.,  0.],
          ...,
          [ 0.,  0.,  0., ...,  3.,  1.,  0.],
          [ 0.,  0.,  0., ...,  0.,  0.,  0.],
          [ 0.,  0.,  0., ...,  0.,  0.,  1.]],
 
         [[ 0.,  0.,  0., ...,  0.,  0.,  0.],
          [ 0.,  0.,  0., ...,  1.,  0.,  0.],
          [ 0.,  0.,  0., ...,  1.,  0.,  0.],
          ...,
          [ 0.,  0.,  0., ...,  0.,  1.,  0.],
          [ 0.,  0.,  0., ...,  2.,  0.,  0.],
          [ 0.,  0.,  0., ...,  0.,  0.,  0.]]],
 
 
        [[[ 0.,  0.,  0., ...,  0.,  0.,  0.],
          [ 0.,  0.,  0., ...,  0.,  0.,  0.],
          [ 0.,  0.,  0., ...,  0.,  0.,  0.],
          ...,
          [ 0.,  0.,  0., ...,  0.,  1.,  2.],
          [ 0.,  0.,  0., ...,  0.,  0.,  0.],
          [ 0.,  0.,  0., ...,  1.,  0.,  1.]],
 
         [[ 0.,  0.,  0., ...,  0.,  0.,  0.],
          [ 0.,  0.,  0., ...,  0.,  0.,  0.],
          [ 0.,  0.,  1., ...,  0.,  0.,  0.],
          ...,
          [ 0.,  0.,  0., ...,  0.,  0.,  0.],
          [ 0.,  0.,  0., ...,  1.,  1.,  0.],
          [ 0.,  0.,  0., ...,  0.,  0.,  1.]]],
 
 
        [[[ 0.,  0.,  0., ...,  0.,  0.,  0.],
          [ 0.,  0.,  0., ...,  1.,  0.,  0.],
          [ 0.,  0.,  0., ...,  1.,  0.,  0.],
          ...,
          [ 0.,  0.,  0., ...,  0.,  2.,  0.],
          [ 0.,  0.,  0., ...,  0.,  0.,  0.],
          [ 0.,  0.,  0., ...,  0.,  0.,  0.]],
 
         [[ 0.,  0.,  0., ...,  0.,  0.,  0.],
          [ 0.,  0.,  0., ...,  1.,  0.,  0.],
          [ 0.,  0.,  0., ...,  2.,  1.,  0.],
          ...,
          [ 0.,  0.,  0., ...,  1.,  0.,  0.],
          [ 0.,  0.,  0., ...,  0.,  0.,  0.],
          [ 0.,  0.,  0., ...,  0.,  0.,  0.]]],
 
 
        ...,
 
 
        [[[ 0.,  0.,  5., ...,  0.,  0.,  0.],
          [ 0.,  2.,  6., ...,  6.,  0.,  0.],
          [ 0.,  5., 22., ..., 14.,  8.,  0.],
          ...,
          [ 0.,  0.,  4., ...,  9.,  5.,  2.],
          [ 0.,  0.,  3., ...,  7.,  3.,  1.],
          [ 0.,  0.,  0., ...,  4.,  6.,  8.]],
 
         [[ 0.,  0.,  2., ...,  0.,  0.,  0.],
          [ 0.,  2.,  2., ...,  9.,  0.,  0.],
          [ 0.,  0., 10., ..., 13.,  7.,  0.],
          ...,
          [ 0.,  0.,  3., ...,  7.,  4.,  1.],
          [ 0.,  0.,  2., ...,  3.,  2.,  2.],
          [ 0.,  0.,  0., ...,  2.,  1.,  1.]]],
 
 
        [[[ 0.,  0.,  5., ...,  0.,  0.,  0.],
          [ 0.,  1.,  4., ...,  5.,  0.,  0.],
          [ 0.,  1.,  7., ...,  3.,  7.,  0.],
          ...,
          [ 0.,  0.,  3., ...,  8.,  1.,  0.],
          [ 0.,  0.,  2., ...,  4.,  0.,  1.],
          [ 0.,  0.,  0., ...,  2.,  4.,  3.]],
 
         [[ 0.,  0.,  2., ...,  0.,  0.,  0.],
          [ 0.,  1.,  5., ...,  7.,  0.,  0.],
          [ 0.,  0.,  8., ...,  3.,  5.,  0.],
          ...,
          [ 0.,  0.,  0., ...,  2.,  1.,  1.],
          [ 0.,  0.,  2., ...,  0.,  1.,  0.],
          [ 0.,  0.,  0., ...,  0.,  0.,  1.]]],
 
 
        [[[ 0.,  0.,  1., ...,  0.,  0.,  0.],
          [ 0.,  0.,  3., ...,  4.,  0.,  0.],
          [ 0.,  2.,  2., ...,  4.,  6.,  0.],
          ...,
          [ 0.,  0.,  0., ..., 12.,  0.,  2.],
          [ 0.,  0.,  1., ...,  3.,  1.,  0.],
          [ 0.,  0.,  0., ...,  2.,  2.,  2.]],
 
         [[ 0.,  0.,  0., ...,  0.,  0.,  0.],
          [ 0.,  0.,  2., ...,  3.,  0.,  0.],
          [ 0.,  0.,  5., ...,  0.,  2.,  0.],
          ...,
          [ 0.,  0.,  0., ...,  1.,  0.,  0.],
          [ 0.,  0.,  1., ...,  1.,  0.,  0.],
          [ 0.,  0.,  0., ...,  2.,  0.,  2.]]]], shape=(4392, 2, 16, 8)),
 array(['2014040101', '2014040102', '2014040103', ..., '2014093022',
        '2014093023', '2014093024'], shape=(4392,), dtype='<U10'))
In [6]:
datasets[1]
Out[6]:
array(['2014040101', '2014040102', '2014040103', ..., '2014093022',
       '2014093023', '2014093024'], shape=(4392,), dtype='<U10')
In [7]:
def extract_timeseries(data: np.ndarray, row: int, col: int, channel: int = 0) -> np.ndarray:
    """
    Extracts the time series of a specific grid location from a 4D numpy array.
    
    Parameters:
        data (np.ndarray): A 4D numpy array with shape (time_steps, channels, height, width)
        row (int): The row index of the desired grid location
        col (int): The column index of the desired grid location
        channel (int): The channel index to extract from (default: 0)
        
    Returns:
        np.ndarray: A 1D array representing the time series at the specified location
    """
    if data.ndim != 4:
        raise ValueError("Input data must be a 4D numpy array (time_steps, channels, height, width)")
    
    if not (0 <= row < data.shape[2]) or not (0 <= col < data.shape[3]):
        raise ValueError("Row or column index out of bounds")
    
    if not (0 <= channel < data.shape[1]):
        raise ValueError("Channel index out of bounds")
    
    return data[:, channel, row, col]

grid_data = datasets[0]

ROW=3
COL=5

# Extract time series from row 5, column 3 in channel 0
time_series = extract_timeseries(grid_data, row=ROW, col=COL, channel=0)

print(time_series)
[0. 1. 0. ... 2. 4. 1.]
In [8]:
len(time_series)
Out[8]:
4392
In [9]:
import pandas as pd
import numpy as np
from datetime import datetime, timedelta
In [10]:
def adjust_timestamps(timestamps):
    """
    Adjusts the given timestamps by subtracting 1 hour from each entry.
    
    Parameters:
        timestamps (numpy array): Array of timestamps in 'YYYYMMDDHH' format.
        
    Returns:
        numpy array: Adjusted timestamps.
    """
    adjusted = []
    for ts in timestamps:
        # Fix invalid '24' hour case
        if ts[-2:] == '24':
            ts = (datetime.strptime(ts[:8], "%Y%m%d") + timedelta(days=1)).strftime("%Y%m%d") + "00"
        dt = datetime.strptime(ts, "%Y%m%d%H") - timedelta(hours=1)
        adjusted.append(dt.strftime("%Y%m%d%H"))
    
    return np.array(adjusted, dtype='<U10')

# Example usage
timestamps = datasets[1]
timestamps = adjust_timestamps(timestamps)
timestamps
Out[10]:
array(['2014040100', '2014040101', '2014040102', ..., '2014093021',
       '2014093022', '2014093023'], shape=(4392,), dtype='<U10')
In [11]:
def format_timestamp(timestamp: str) -> str:
    """
    Converts timestamp from 'YYYYMMDDHH' to 'YYYY-MM-DD HH:00'.
    """
    dt = datetime.strptime(timestamp, "%Y%m%d%H")
    return dt.strftime("%Y-%m-%d %H:00")

def create_dataframe(timeseries: np.ndarray, timestamps: np.ndarray) -> pd.DataFrame:
    if timeseries.shape[0] != timestamps.shape[0]:
        raise ValueError("Time series and timestamps must have the same length")

    formatted_timestamps = [format_timestamp(ts) for ts in timestamps]

    df = pd.DataFrame({
        'timestamp': formatted_timestamps,
        'value': timeseries
    })
    return df


df = create_dataframe(time_series, timestamps)
print(df)
             timestamp  value
0     2014-04-01 00:00    0.0
1     2014-04-01 01:00    1.0
2     2014-04-01 02:00    0.0
3     2014-04-01 03:00    0.0
4     2014-04-01 04:00    1.0
...                ...    ...
4387  2014-09-30 19:00   16.0
4388  2014-09-30 20:00   10.0
4389  2014-09-30 21:00    2.0
4390  2014-09-30 22:00    4.0
4391  2014-09-30 23:00    1.0

[4392 rows x 2 columns]

Get the last 10 days for validation, like in the paper (+5 as input):

In [12]:
train_data_df = df[df['timestamp']<'2014-09-21 00:00']
In [13]:
test_data_df = df.copy()
In [14]:
df = df[df['timestamp']>='2014-09-16 00:00']

TimeGPT¶

In [15]:
from nixtla import NixtlaClient
In [ ]:
nixtla_client = NixtlaClient(
    api_key = "nixak-"
)
In [18]:
nixtla_client.validate_api_key()
INFO:nixtla.nixtla_client:Happy Forecasting! :)
Out[18]:
True
In [19]:
nixtla_client.plot(df, time_col='timestamp', target_col='value')
Out[19]:
No description has been provided for this image

Experiment v0¶

Simple forecast¶
In [123]:
timegpt_fcst_df = nixtla_client.forecast(df=df, h=24, freq='h', time_col='timestamp', target_col='value')
timegpt_fcst_df
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Restricting input...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
Out[123]:
timestamp TimeGPT
0 2014-10-01 00:00:00 -1.202475
1 2014-10-01 01:00:00 -0.848185
2 2014-10-01 02:00:00 -0.593907
3 2014-10-01 03:00:00 -1.358113
4 2014-10-01 04:00:00 -2.240385
5 2014-10-01 05:00:00 -1.024400
6 2014-10-01 06:00:00 9.498710
7 2014-10-01 07:00:00 28.237295
8 2014-10-01 08:00:00 53.401657
9 2014-10-01 09:00:00 40.392822
10 2014-10-01 10:00:00 16.469150
11 2014-10-01 11:00:00 9.792094
12 2014-10-01 12:00:00 12.540842
13 2014-10-01 13:00:00 13.419383
14 2014-10-01 14:00:00 12.166123
15 2014-10-01 15:00:00 10.900003
16 2014-10-01 16:00:00 12.008976
17 2014-10-01 17:00:00 17.562895
18 2014-10-01 18:00:00 18.652906
19 2014-10-01 19:00:00 14.093687
20 2014-10-01 20:00:00 8.132493
21 2014-10-01 21:00:00 4.112089
22 2014-10-01 22:00:00 3.221751
23 2014-10-01 23:00:00 2.066868
In [124]:
nixtla_client.plot(df, timegpt_fcst_df, time_col='timestamp', target_col='value')
Out[124]:
No description has been provided for this image
Historical forecast¶
In [ ]:
timegpt_fcst_with_history_df = nixtla_client.forecast(
    df=df, h=1, time_col='timestamp', target_col='value', freq='h',
    add_history=True,
)
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
In [31]:
timegpt_fcst_with_history_df.to_csv(f'predictions/timegpt/fcst_{ROW}_{COL}_h{HORIZON}.csv')
In [32]:
nixtla_client.plot(df, timegpt_fcst_with_history_df, time_col='timestamp', target_col='value')
Out[32]:
No description has been provided for this image
In [29]:
def compute_rmse(original_df, forecast_df):
    # Convert timestamp to datetime
    original_df['timestamp'] = pd.to_datetime(original_df['timestamp'])
    forecast_df['timestamp'] = pd.to_datetime(forecast_df['timestamp'])

    # Merge the DataFrames based on the timestamp
    merged_df = pd.merge(original_df, forecast_df, on='timestamp', how='inner')
    print(merged_df)

    # Calculate the squared differences
    merged_df['squared_diff'] = (merged_df['value'] - merged_df['TimeGPT'])**2

    # Calculate RMSE
    rmse = np.sqrt(merged_df['squared_diff'].mean())
    return rmse
In [ ]:
rmse = compute_rmse(df,timegpt_fcst_with_history_df)
print(f"RMSE: {rmse}")
Run forecasts for all grid cells and store results¶
In [94]:
import os.path

def prediction_exists(row, col):
    return os.path.isfile(f'predictions/timegpt/fcst_{row}_{col}.csv')

prediction_exists(2,1)
Out[94]:
False
In [97]:
def load_prediction(row,col):
    timegpt_fcst_with_history_df = pd.read_csv(f'predictions/timegpt/fcst_{row}_{col}.csv')
    return timegpt_fcst_with_history_df

load_prediction(1,3)
Out[97]:
timestamp TimeGPT
0 2014-09-21 00:00:00 16.648037
1 2014-09-21 01:00:00 10.752392
2 2014-09-21 02:00:00 6.054309
3 2014-09-21 03:00:00 2.488530
4 2014-09-21 04:00:00 1.055259
... ... ...
236 2014-09-30 20:00:00 28.864220
237 2014-09-30 21:00:00 26.119790
238 2014-09-30 22:00:00 21.338356
239 2014-09-30 23:00:00 13.381630
240 2014-10-01 00:00:00 9.842307

241 rows × 2 columns

In [105]:
def compute_cell_rmse(row, col):
    time_series = extract_timeseries(grid_data, row=row, col=col, channel=0)
    timestamps = datasets[1]
    timestamps = adjust_timestamps(timestamps)
    df = create_dataframe(time_series, timestamps)
    df = df[df['timestamp']>='2014-09-16 00:00']

    if prediction_exists(row, col):
        print('Loading predictions ...')
        timegpt_fcst_with_history_df = load_prediction(row, col)
    else:
        print('Computing predicitons ...')
        timegpt_fcst_with_history_df = nixtla_client.forecast(
            df=df, h=1, time_col='timestamp', target_col='value', freq='h',
            add_history=True,
        )
        timegpt_fcst_with_history_df.to_csv(f'predictions/timegpt/fcst_{row}_{col}.csv', index=False)
    rmse = compute_rmse(df, timegpt_fcst_with_history_df)
    print(f"[{row},{col}] RMSE:{rmse}")
    return rmse

rmses =  []

for row in range(0,16):
    for col in range(0,8):
        rmses.append(compute_cell_rmse(row,col))

print(rmses)
Loading predictions ...
[0,0] RMSE:0.002570502930800306
Loading predictions ...
[0,1] RMSE:0.002570502930800306
Loading predictions ...
[0,2] RMSE:3.5633947837540783
Loading predictions ...
[0,3] RMSE:10.037934923310697
Loading predictions ...
[0,4] RMSE:15.239814706642761
Loading predictions ...
[0,5] RMSE:0.002570502930800306
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
Loading predictions ...
[0,6] RMSE:0.002570502930800306
Loading predictions ...
[0,7] RMSE:0.002570502930800306
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[1,0] RMSE:0.002570502930800306
Loading predictions ...
[1,1] RMSE:6.072188987583208
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[1,2] RMSE:6.10467960056596
Loading predictions ...
[1,3] RMSE:10.603164877491775
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[1,4] RMSE:13.917665751860108
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[1,5] RMSE:11.204286010693234
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[1,6] RMSE:2.810509161338358
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[1,7] RMSE:0.0025654494710700606
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[2,0] RMSE:0.0025654494710700606
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[2,1] RMSE:3.094132360384688
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[2,2] RMSE:8.124441799259046
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[2,3] RMSE:14.124176681319927
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[2,4] RMSE:9.87947194144156
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[2,5] RMSE:19.116945715244864
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[2,6] RMSE:4.0930457339758926
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[2,7] RMSE:0.0025654494710700606
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[3,0] RMSE:0.002570502930800306
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[3,1] RMSE:9.823857990415844
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[3,2] RMSE:13.609803980126207
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[3,3] RMSE:19.39126985581633
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[3,4] RMSE:24.82940198763119
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[3,5] RMSE:8.01946406849938
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[3,6] RMSE:0.002570502930800306
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[3,7] RMSE:0.0025654494710700606
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[4,0] RMSE:0.0025654494710700606
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[4,1] RMSE:15.353709343653744
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[4,2] RMSE:13.703235791015969
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[4,3] RMSE:13.525093211179996
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[4,4] RMSE:9.825678636444476
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[4,5] RMSE:7.809890565314297
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[4,6] RMSE:0.0025654494710700606
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[4,7] RMSE:0.0025654494710700606
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[5,0] RMSE:0.002570502930800306
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[5,1] RMSE:18.817138848097564
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[5,2] RMSE:17.183880967133586
Loading predictions ...
[5,3] RMSE:19.97543575145462
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[5,4] RMSE:14.245502307836027
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[5,5] RMSE:0.0025654494710700606
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[5,6] RMSE:0.0025654494710700606
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[5,7] RMSE:0.002570502930800306
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[6,0] RMSE:0.0025654494710700606
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[6,1] RMSE:16.935044642220817
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[6,2] RMSE:18.79578174157705
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[6,3] RMSE:20.080684570699418
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[6,4] RMSE:11.641629805276798
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[6,5] RMSE:0.002570502930800306
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[6,6] RMSE:0.0025654494710700606
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[6,7] RMSE:0.002570502930800306
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[7,0] RMSE:0.0025654494710700606
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[7,1] RMSE:13.642025157738148
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[7,2] RMSE:17.016431550912763
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[7,3] RMSE:21.37184891958808
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[7,4] RMSE:8.20325748668239
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[7,5] RMSE:0.002570502930800306
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[7,6] RMSE:0.0025654494710700606
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[7,7] RMSE:0.0025654494710700606
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[8,0] RMSE:6.403986488399714
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[8,1] RMSE:11.873086183839364
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[8,2] RMSE:12.35191115935829
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[8,3] RMSE:15.479802706277466
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[8,4] RMSE:8.601080918207966
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[8,5] RMSE:1.547153580097292
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[8,6] RMSE:0.002570502930800306
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[8,7] RMSE:0.0025654494710700606
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[9,0] RMSE:24.094423827936296
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[9,1] RMSE:8.012939328699487
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[9,2] RMSE:8.77825577110615
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[9,3] RMSE:7.686780244079536
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[9,4] RMSE:5.639365513049377
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[9,5] RMSE:0.0025654494710700606
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[9,6] RMSE:4.854884546519325
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[9,7] RMSE:0.002570502930800306
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[10,0] RMSE:14.313091700592
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[10,1] RMSE:9.821399516794514
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[10,2] RMSE:6.817657614659597
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[10,3] RMSE:5.287142657538004
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[10,4] RMSE:3.880122899223147
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[10,5] RMSE:0.002570502930800306
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[10,6] RMSE:5.188028338714668
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[10,7] RMSE:2.1259317656706505
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[11,0] RMSE:13.488011454485571
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[11,1] RMSE:12.72174381434626
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[11,2] RMSE:0.0025654494710700606
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[11,3] RMSE:0.0025654494710700606
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[11,4] RMSE:0.0025654494710700606
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[11,5] RMSE:1.6790453164840837
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[11,6] RMSE:0.9429252848834933
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[11,7] RMSE:0.0025654494710700606
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[12,0] RMSE:7.752473567223986
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[12,1] RMSE:0.0025654494710700606
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[12,2] RMSE:5.74673617862669
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[12,3] RMSE:9.750837264686353
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[12,4] RMSE:2.5531225667590034
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[12,5] RMSE:1.8957608947336775
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[12,6] RMSE:0.002570502930800306
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[12,7] RMSE:0.0025654494710700606
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[13,0] RMSE:0.0025654494710700606
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[13,1] RMSE:0.002570502930800306
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[13,2] RMSE:2.9458867986620376
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[13,3] RMSE:7.710715412667907
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[13,4] RMSE:1.6876675342706142
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[13,5] RMSE:3.9700305727035357
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[13,6] RMSE:1.8811816927461884
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[13,7] RMSE:1.4560359282046909
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[14,0] RMSE:0.0025654494710700606
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[14,1] RMSE:0.002570502930800306
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[14,2] RMSE:3.0911829947417
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[14,3] RMSE:4.0605317926657
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[14,4] RMSE:4.7903847224501765
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[14,5] RMSE:3.0232607390711816
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[14,6] RMSE:2.015987717342668
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[14,7] RMSE:0.9634530333874449
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[15,0] RMSE:0.002570502930800306
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[15,1] RMSE:0.0025654494710700606
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[15,2] RMSE:0.0025654494710700606
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[15,3] RMSE:0.002570502930800306
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[15,4] RMSE:4.263672605310704
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[15,5] RMSE:1.470171475186865
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
[15,6] RMSE:2.2082179371533774
Computing predicitons ...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
[15,7] RMSE:2.0426881318865613
[np.float64(0.002570502930800306), np.float64(0.002570502930800306), np.float64(3.5633947837540783), np.float64(10.037934923310697), np.float64(15.239814706642761), np.float64(0.002570502930800306), np.float64(0.002570502930800306), np.float64(0.002570502930800306), np.float64(0.002570502930800306), np.float64(6.072188987583208), np.float64(6.10467960056596), np.float64(10.603164877491775), np.float64(13.917665751860108), np.float64(11.204286010693234), np.float64(2.810509161338358), np.float64(0.0025654494710700606), np.float64(0.0025654494710700606), np.float64(3.094132360384688), np.float64(8.124441799259046), np.float64(14.124176681319927), np.float64(9.87947194144156), np.float64(19.116945715244864), np.float64(4.0930457339758926), np.float64(0.0025654494710700606), np.float64(0.002570502930800306), np.float64(9.823857990415844), np.float64(13.609803980126207), np.float64(19.39126985581633), np.float64(24.82940198763119), np.float64(8.01946406849938), np.float64(0.002570502930800306), np.float64(0.0025654494710700606), np.float64(0.0025654494710700606), np.float64(15.353709343653744), np.float64(13.703235791015969), np.float64(13.525093211179996), np.float64(9.825678636444476), np.float64(7.809890565314297), np.float64(0.0025654494710700606), np.float64(0.0025654494710700606), np.float64(0.002570502930800306), np.float64(18.817138848097564), np.float64(17.183880967133586), np.float64(19.97543575145462), np.float64(14.245502307836027), np.float64(0.0025654494710700606), np.float64(0.0025654494710700606), np.float64(0.002570502930800306), np.float64(0.0025654494710700606), np.float64(16.935044642220817), np.float64(18.79578174157705), np.float64(20.080684570699418), np.float64(11.641629805276798), np.float64(0.002570502930800306), np.float64(0.0025654494710700606), np.float64(0.002570502930800306), np.float64(0.0025654494710700606), np.float64(13.642025157738148), np.float64(17.016431550912763), np.float64(21.37184891958808), np.float64(8.20325748668239), np.float64(0.002570502930800306), np.float64(0.0025654494710700606), np.float64(0.0025654494710700606), np.float64(6.403986488399714), np.float64(11.873086183839364), np.float64(12.35191115935829), np.float64(15.479802706277466), np.float64(8.601080918207966), np.float64(1.547153580097292), np.float64(0.002570502930800306), np.float64(0.0025654494710700606), np.float64(24.094423827936296), np.float64(8.012939328699487), np.float64(8.77825577110615), np.float64(7.686780244079536), np.float64(5.639365513049377), np.float64(0.0025654494710700606), np.float64(4.854884546519325), np.float64(0.002570502930800306), np.float64(14.313091700592), np.float64(9.821399516794514), np.float64(6.817657614659597), np.float64(5.287142657538004), np.float64(3.880122899223147), np.float64(0.002570502930800306), np.float64(5.188028338714668), np.float64(2.1259317656706505), np.float64(13.488011454485571), np.float64(12.72174381434626), np.float64(0.0025654494710700606), np.float64(0.0025654494710700606), np.float64(0.0025654494710700606), np.float64(1.6790453164840837), np.float64(0.9429252848834933), np.float64(0.0025654494710700606), np.float64(7.752473567223986), np.float64(0.0025654494710700606), np.float64(5.74673617862669), np.float64(9.750837264686353), np.float64(2.5531225667590034), np.float64(1.8957608947336775), np.float64(0.002570502930800306), np.float64(0.0025654494710700606), np.float64(0.0025654494710700606), np.float64(0.002570502930800306), np.float64(2.9458867986620376), np.float64(7.710715412667907), np.float64(1.6876675342706142), np.float64(3.9700305727035357), np.float64(1.8811816927461884), np.float64(1.4560359282046909), np.float64(0.0025654494710700606), np.float64(0.002570502930800306), np.float64(3.0911829947417), np.float64(4.0605317926657), np.float64(4.7903847224501765), np.float64(3.0232607390711816), np.float64(2.015987717342668), np.float64(0.9634530333874449), np.float64(0.002570502930800306), np.float64(0.0025654494710700606), np.float64(0.0025654494710700606), np.float64(0.002570502930800306), np.float64(4.263672605310704), np.float64(1.470171475186865), np.float64(2.2082179371533774), np.float64(2.0426881318865613)]

image.png

In [109]:
np.mean(rmses)
Out[109]:
np.float64(5.7716825988548335)

Better than ST-ResNet?

image.png

Identified issues¶

https://docs.nixtla.io/docs/tutorials-historical_forecast

When add_history is set to True, the output DataFrame will include not only the future forecasts determined by the h argument, but also the historical predictions. Currently, the historical forecasts are not affected by h, and have a fix horizon depending on the frequency of the data. The historical forecasts are produced in a rolling window fashion, and concatenated. This means that the model is applied sequentially at each time step using only the most recent information available up to that point.

Experiment v1¶

Multiple cells at once¶
In [20]:
def get_df(row, col, i=0, channel=0):
    time_series = extract_timeseries(grid_data, row=row, col=col, channel=channel)
    timestamps = datasets[1]
    timestamps = adjust_timestamps(timestamps)
    df = create_dataframe(time_series, timestamps)
    df['unique_id'] = i
    return df
In [21]:
dfs = []
i=0

for channel in [0,1]:
    for row, col in [(3,5),(4,5)]:
        dfs.append(get_df(row,col,i,channel))
        i+=1

df = pd.concat(dfs, axis=0)
df
Out[21]:
timestamp value unique_id
0 2014-04-01 00:00 0.0 0
1 2014-04-01 01:00 1.0 0
2 2014-04-01 02:00 0.0 0
3 2014-04-01 03:00 0.0 0
4 2014-04-01 04:00 1.0 0
... ... ... ...
4387 2014-09-30 19:00 29.0 3
4388 2014-09-30 20:00 8.0 3
4389 2014-09-30 21:00 6.0 3
4390 2014-09-30 22:00 7.0 3
4391 2014-09-30 23:00 5.0 3

17568 rows × 3 columns

In [22]:
nixtla_client.plot(df, time_col='timestamp', target_col='value')
Out[22]:
No description has been provided for this image
In [23]:
timegpt_fcst_df = nixtla_client.forecast(df=df, h=24, freq='h', time_col='timestamp', target_col='value')
timegpt_fcst_df
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Querying model metadata...
INFO:nixtla.nixtla_client:Restricting input...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
Out[23]:
unique_id timestamp TimeGPT
0 0 2014-10-01 00:00:00 -1.202614
1 0 2014-10-01 01:00:00 -0.847775
2 0 2014-10-01 02:00:00 -0.594713
3 0 2014-10-01 03:00:00 -1.358931
4 0 2014-10-01 04:00:00 -2.239870
... ... ... ...
91 3 2014-10-01 19:00:00 25.792120
92 3 2014-10-01 20:00:00 12.507367
93 3 2014-10-01 21:00:00 8.102301
94 3 2014-10-01 22:00:00 6.962522
95 3 2014-10-01 23:00:00 5.438268

96 rows × 3 columns

In [24]:
nixtla_client.plot(df, timegpt_fcst_df, time_col='timestamp', target_col='value', max_insample_length=24*11)
Out[24]:
No description has been provided for this image
Backtesting¶
In [67]:
def backtest_forecasting(df, start_timestamp, horizon, timestamp='ds', target_col='y'):
    """
    Backtest a time forecasting model by iteratively making forecasts at fixed intervals.
    
    :param df: DataFrame with time series data (must contain timestamp and target_col columns)
    :param start_timestamp: The timestamp from which to start the backtesting process
    :param horizon: Forecasting horizon (number of time steps to forecast at each iteration)
    
    :return: A DataFrame with concatenated forecasts
    """
    
    # Ensure timestamp is in datetime format
    df[timestamp] = pd.to_datetime(df[timestamp])
    
    all_forecasts = []
    current_timestamp = start_timestamp

    print(f'current: {current_timestamp}')
    print(f'max: {df[timestamp].max()}')
    
    while current_timestamp < df[timestamp].max():
        # Filter the data up to the current timestamp

        print(f'current: {current_timestamp}')
        df_subset = df[df[timestamp] < current_timestamp]
        
        # Make a forecast using the model
        forecast_df = nixtla_client.forecast(df=df_subset, h=horizon, freq='h', time_col=timestamp, target_col=target_col)
        
        # Append the results
        all_forecasts.append(forecast_df)
        
        # Move to the next forecasting window
        current_timestamp += pd.Timedelta(hours=horizon)
    
    # Concatenate all forecasts into a single DataFrame
    return pd.concat(all_forecasts, ignore_index=True)
In [ ]:
timegpt_fcst_df_backtest = backtest_forecasting(df, datetime(2014,9,21), 24, timestamp='timestamp', target_col='value')
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Restricting input...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
current: 2014-09-21 00:00:00
max: 2014-09-30 23:00:00
current: 2014-09-21 00:00:00
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Restricting input...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
current: 2014-09-22 00:00:00
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Restricting input...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
current: 2014-09-23 00:00:00
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Restricting input...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
current: 2014-09-24 00:00:00
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Restricting input...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
current: 2014-09-25 00:00:00
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Restricting input...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
current: 2014-09-26 00:00:00
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Restricting input...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
current: 2014-09-27 00:00:00
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Restricting input...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
current: 2014-09-28 00:00:00
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Restricting input...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
current: 2014-09-29 00:00:00
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Restricting input...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
current: 2014-09-30 00:00:00
In [27]:
nixtla_client.plot(df, timegpt_fcst_df_backtest, time_col='timestamp', target_col='value', max_insample_length=24*11)
Out[27]:
No description has been provided for this image
In [30]:
rmse = compute_rmse(df, timegpt_fcst_df_backtest)
rmse
               timestamp  value  unique_id_x  unique_id_y   TimeGPT
0    2014-09-21 01:00:00    4.0            0            0  2.516364
1    2014-09-21 01:00:00    4.0            0            1 -0.000597
2    2014-09-21 01:00:00    4.0            0            2  0.754800
3    2014-09-21 01:00:00    4.0            0            3 -1.170985
4    2014-09-21 02:00:00    1.0            0            0  1.462680
...                  ...    ...          ...          ...       ...
3819 2014-09-30 22:00:00    7.0            3            3  2.302446
3820 2014-09-30 23:00:00    5.0            3            0  5.505838
3821 2014-09-30 23:00:00    5.0            3            1  4.514214
3822 2014-09-30 23:00:00    5.0            3            2  3.481431
3823 2014-09-30 23:00:00    5.0            3            3  2.236673

[3824 rows x 5 columns]
Out[30]:
np.float64(8.346675647024847)
In [ ]:
timegpt_fcst_df_backtest = backtest_forecasting(df, datetime(2014,9,21), 48, timestamp='timestamp', target_col='value')
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
WARNING:nixtla.nixtla_client:The specified horizon "h" exceeds the model horizon, this may lead to less accurate forecasts. Please consider using a smaller horizon.
INFO:nixtla.nixtla_client:Restricting input...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
current: 2014-09-21 00:00:00
max: 2014-09-30 23:00:00
current: 2014-09-21 00:00:00
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
WARNING:nixtla.nixtla_client:The specified horizon "h" exceeds the model horizon, this may lead to less accurate forecasts. Please consider using a smaller horizon.
INFO:nixtla.nixtla_client:Restricting input...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
current: 2014-09-23 00:00:00
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
WARNING:nixtla.nixtla_client:The specified horizon "h" exceeds the model horizon, this may lead to less accurate forecasts. Please consider using a smaller horizon.
INFO:nixtla.nixtla_client:Restricting input...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
current: 2014-09-25 00:00:00
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
WARNING:nixtla.nixtla_client:The specified horizon "h" exceeds the model horizon, this may lead to less accurate forecasts. Please consider using a smaller horizon.
INFO:nixtla.nixtla_client:Restricting input...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
current: 2014-09-27 00:00:00
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
WARNING:nixtla.nixtla_client:The specified horizon "h" exceeds the model horizon, this may lead to less accurate forecasts. Please consider using a smaller horizon.
INFO:nixtla.nixtla_client:Restricting input...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
current: 2014-09-29 00:00:00
In [32]:
nixtla_client.plot(df, timegpt_fcst_df_backtest, time_col='timestamp', target_col='value', max_insample_length=24*11)
Out[32]:
No description has been provided for this image
In [33]:
rmse = compute_rmse(df, timegpt_fcst_df_backtest)
rmse
               timestamp  value  unique_id_x  unique_id_y   TimeGPT
0    2014-09-21 01:00:00    4.0            0            0  2.516364
1    2014-09-21 01:00:00    4.0            0            1 -0.000597
2    2014-09-21 01:00:00    4.0            0            2  0.754800
3    2014-09-21 01:00:00    4.0            0            3 -1.170985
4    2014-09-21 02:00:00    1.0            0            0  1.462680
...                  ...    ...          ...          ...       ...
3819 2014-09-30 22:00:00    7.0            3            3  2.953221
3820 2014-09-30 23:00:00    5.0            3            0  3.396904
3821 2014-09-30 23:00:00    5.0            3            1  4.583953
3822 2014-09-30 23:00:00    5.0            3            2  2.344816
3823 2014-09-30 23:00:00    5.0            3            3  2.392262

[3824 rows x 5 columns]
Out[33]:
np.float64(8.666202218206987)
All cells & backtesting¶
In [56]:
def compute_rmse(original_df, forecast_df, timestamp='ds', target_col='y', predict_col='TimeGPT', unique_id='unique_id'):
    # Convert timestamp to datetime
    original_df[timestamp] = pd.to_datetime(original_df[timestamp])
    forecast_df[timestamp] = pd.to_datetime(forecast_df[timestamp])

    # Merge the DataFrames based on the timestamp
    merged_df = pd.merge(original_df, forecast_df, on=[timestamp,unique_id], how='inner')
    #print(merged_df)

    # Calculate the squared differences
    merged_df['squared_diff'] = (merged_df[target_col] - merged_df[predict_col])**2

    # Calculate RMSE
    rmse = np.sqrt(merged_df['squared_diff'].mean())
    return rmse
In [35]:
dfs = []
i=0

for channel in [0,1]:
    for row in range(0,16):
        for col in range(0,8):
            dfs.append(get_df(row,col,i,channel))
            i+=1

df = pd.concat(dfs, axis=0)
df
Out[35]:
timestamp value unique_id
0 2014-04-01 00:00 0.0 0
1 2014-04-01 01:00 0.0 0
2 2014-04-01 02:00 0.0 0
3 2014-04-01 03:00 0.0 0
4 2014-04-01 04:00 0.0 0
... ... ... ...
4387 2014-09-30 19:00 1.0 255
4388 2014-09-30 20:00 4.0 255
4389 2014-09-30 21:00 1.0 255
4390 2014-09-30 22:00 1.0 255
4391 2014-09-30 23:00 2.0 255

1124352 rows × 3 columns

In [ ]:
import os 

HORIZON = 1
path = f'predictions/timegpt/fcst_c01_h{HORIZON}.csv'

if os.path.isfile(path):
    timegpt_fcst_df_backtest = pd.read_csv(path)
else: 
    timegpt_fcst_df_backtest = backtest_forecasting(df, datetime(2014,9,21), HORIZON, timestamp='timestamp', target_col='value')
    timegpt_fcst_df_backtest.to_csv(path, index=False)

rmse = compute_rmse(df, timegpt_fcst_df_backtest, timestamp='timestamp', target_col='value')
print(f'horizon={HORIZON}: rmse={rmse}')
                timestamp  value  unique_id   TimeGPT
0     2014-09-21 01:00:00    0.0          0  0.003328
1     2014-09-21 02:00:00    0.0          0  0.003328
2     2014-09-21 03:00:00    0.0          0  0.003328
3     2014-09-21 04:00:00    0.0          0  0.003328
4     2014-09-21 05:00:00    0.0          0  0.003328
...                   ...    ...        ...       ...
61179 2014-09-30 19:00:00    1.0        255  3.210955
61180 2014-09-30 20:00:00    4.0        255  1.871202
61181 2014-09-30 21:00:00    1.0        255  2.555601
61182 2014-09-30 22:00:00    1.0        255  0.832277
61183 2014-09-30 23:00:00    2.0        255  0.272251

[61184 rows x 4 columns]
horizon=1: rmse=5.695230340297138
  • horizon=1: rmse=5.695230340297138
  • horizon=12: rmse=7.623156119092268
  • horizon=24: rmse=8.931978485071902
In [42]:
nixtla_client.plot(df, timegpt_fcst_df_backtest, time_col='timestamp', target_col='value', max_insample_length=24*11, unique_ids=[22,36,51,60,86,96,108,127])
Out[42]:
No description has been provided for this image

image-2.png

Results¶

Extended Table 3: Comparisons with baselines on BikeNYC. The results of ARIMA, SARIMA, VAR, and 4 DeepST variants are taken from (Zhang et al. 2016). ST-ResNet results are taken from (Zhang et al. 2017).

Model RMSE
ARIMA 10.56
SARIMA 10.07
VAR 9.92
DeepST-C 8.39
DeepST-CP 7.64
DeepST-CPT 7.56
DeepST-CPTM 7.43
ST-ResNet 6.33
TimeGPT (horizon=1) 5.70
TimeGPT (horizon=12) 7.62
TimeGPT (horizon=24) 8.93

image.png

Conclusion¶

Since the BikeNYC dataset has been public for a long time, we must assume that it may have been used to train TimeGPT. This means that the foundation model may have seen the solution before. (Similar to how ChatGPT performs well on some standardized tests because the tests incl answers are in its training set.)

For future experiments, we should therefore find a novel dataset that the model cannot have seen before.

BikeVienna Experiment¶

This is a newer dataset from Vienna, Austria. It's possible that this is also included in the training data but it is less likely.

In [3]:
import pandas as pd
from nixtla import NixtlaClient
In [ ]:
nixtla_client = NixtlaClient(
    api_key = "nixak-"
)
nixtla_client.validate_api_key()
INFO:nixtla.nixtla_client:Happy Forecasting! :)
Out[ ]:
True
In [26]:
df = pd.read_csv('datasets/BikeVienna/service_2-2019-05-07_2019-12-31.csv', usecols=['timestamp','station_id','vehicles_available'])
df.rename(columns={'timestamp':'ds','station_id':'unique_id', 'vehicles_available':'y'}, inplace=True)
df['ds'] = pd.to_datetime(df['ds']).dt.tz_convert(None)
df
Out[26]:
ds unique_id y
0 2019-05-06 22:07:01 53 4
1 2019-05-06 22:24:01 53 4
2 2019-05-06 22:37:01 53 4
3 2019-05-06 22:52:01 53 4
4 2019-05-06 23:07:01 53 4
... ... ... ...
2538844 2019-12-31 21:52:01 128 22
2538845 2019-12-31 22:07:01 128 22
2538846 2019-12-31 22:24:01 128 22
2538847 2019-12-31 22:37:01 128 22
2538848 2019-12-31 22:52:01 128 22

2538849 rows × 3 columns

Cutting off autumn & winter months:

In [27]:
df = df[df.ds<'2019-08-15']
df
Out[27]:
ds unique_id y
0 2019-05-06 22:07:01 53 4
1 2019-05-06 22:24:01 53 4
2 2019-05-06 22:37:01 53 4
3 2019-05-06 22:52:01 53 4
4 2019-05-06 23:07:01 53 4
... ... ... ...
2527148 2019-08-14 22:52:01 128 6
2527149 2019-08-14 23:07:01 128 11
2527150 2019-08-14 23:24:01 128 11
2527151 2019-08-14 23:37:01 128 11
2527152 2019-08-14 23:52:01 128 11

1138855 rows × 3 columns

In [28]:
len(df.unique_id.unique())
Out[28]:
120
In [29]:
nixtla_client.plot(df, max_insample_length=2400)
Out[29]:
No description has been provided for this image

Resampling¶

In [30]:
df = df.set_index('ds').groupby('unique_id').resample('60min')['y'].max()
df.ffill(inplace=True)
df = df.reset_index()
df
Out[30]:
unique_id ds y
0 9 2019-05-06 22:00:00 8.0
1 9 2019-05-06 23:00:00 8.0
2 9 2019-05-07 00:00:00 8.0
3 9 2019-05-07 01:00:00 8.0
4 9 2019-05-07 02:00:00 8.0
... ... ... ...
288235 128 2019-08-14 19:00:00 5.0
288236 128 2019-08-14 20:00:00 4.0
288237 128 2019-08-14 21:00:00 4.0
288238 128 2019-08-14 22:00:00 6.0
288239 128 2019-08-14 23:00:00 11.0

288240 rows × 3 columns

In [31]:
nixtla_client.plot(df, max_insample_length=2400)
Out[31]:
No description has been provided for this image

Baseline¶

In [38]:
from statsforecast import StatsForecast
from statsforecast.models import (
    HistoricAverage,
    SeasonalNaive,
    AutoARIMA
)
In [42]:
models = [
    HistoricAverage(),
    SeasonalNaive(season_length=24),
    AutoARIMA(),
    
]
In [ ]:
sf = StatsForecast( 
    models=models,
    freq='h', 
    fallback_model = SeasonalNaive(season_length=7),
    n_jobs=-1,
)
In [61]:
forecasts_df = sf.forecast(df=df, h=24, level=[])
forecasts_df
Out[61]:
unique_id ds HistoricAverage SeasonalNaive AutoARIMA
0 9 2019-08-15 00:00:00 14.399667 7.0 17.722890
1 9 2019-08-15 01:00:00 14.399667 5.0 17.505003
2 9 2019-08-15 02:00:00 14.399667 5.0 17.245954
3 9 2019-08-15 03:00:00 14.399667 6.0 17.099813
4 9 2019-08-15 04:00:00 14.399667 6.0 17.014962
... ... ... ... ... ...
2875 128 2019-08-15 19:00:00 15.333472 5.0 9.244525
2876 128 2019-08-15 20:00:00 15.333472 4.0 9.435932
2877 128 2019-08-15 21:00:00 15.333472 4.0 9.637041
2878 128 2019-08-15 22:00:00 15.333472 6.0 9.782622
2879 128 2019-08-15 23:00:00 15.333472 11.0 9.941062

2880 rows × 5 columns

In [62]:
nixtla_client.plot(df, forecasts_df, max_insample_length=24*11)
Out[62]:
No description has been provided for this image
In [63]:
def backtest_baselines(df, start_timestamp, horizon, timestamp='ds', target_col='y'):
    df[timestamp] = pd.to_datetime(df[timestamp])
    
    all_forecasts = []
    current_timestamp = start_timestamp

    print(f'current: {current_timestamp}')
    print(f'max: {df[timestamp].max()}')
    
    while current_timestamp < df[timestamp].max():
        print(f'current: {current_timestamp}')
        df_subset = df[df[timestamp] <= current_timestamp]
        
        forecast_df = sf.forecast(df=df_subset, h=horizon, level=[], time_col=timestamp, target_col=target_col)
        all_forecasts.append(forecast_df)
        
        current_timestamp += pd.Timedelta(hours=horizon)
    
    return pd.concat(all_forecasts, ignore_index=True)
In [64]:
import os
import numpy as np
from datetime import datetime
In [77]:
HORIZON = 12
path = f'predictions/timegpt/baselines_vienna_h{HORIZON}.csv'

if os.path.isfile(path):
    baseline_df_backtest = pd.read_csv(path)
else: 
    baseline_df_backtest = backtest_baselines(df, datetime(2019,8,5), HORIZON)
    baseline_df_backtest.to_csv(path, index=False)

for model_name in ['HistoricAverage','SeasonalNaive','AutoARIMA']:
    rmse = compute_rmse(df, baseline_df_backtest, predict_col=model_name)		
    print(f'{model_name} horizon={HORIZON}: rmse={rmse}')
HistoricAverage horizon=12: rmse=7.035121951753955
SeasonalNaive horizon=12: rmse=7.870333843727055
AutoARIMA horizon=12: rmse=5.2139226565300625
  • HistoricAverage horizon=1: rmse=7.0228811038923284
  • SeasonalNaive horizon=1: rmse=7.870333843727055
  • AutoARIMA horizon=1: rmse=2.263917097458598
  • HistoricAverage horizon=12: rmse=7.019542982157326
  • SeasonalNaive horizon=12: rmse=7.731746974655947
  • AutoARIMA horizon=12: rmse=5.150516932729188
  • HistoricAverage horizon=24: rmse=7.042648290307221
  • SeasonalNaive horizon=24: rmse=7.870333843727055
  • AutoARIMA horizon=24: rmse=6.38808025709064
In [78]:
nixtla_client.plot(df, baseline_df_backtest, max_insample_length=24*11)
Out[78]:
No description has been provided for this image

TimeGPT¶

Simple forecast¶

In [69]:
tmp = df[df.unique_id==16]

timegpt_fcst_df = nixtla_client.forecast(df=tmp, h=24, freq='h')
timegpt_fcst_df
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Restricting input...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
Out[69]:
unique_id ds TimeGPT
0 16 2019-12-31 23:00:00 17.982372
1 16 2020-01-01 00:00:00 17.792389
2 16 2020-01-01 01:00:00 17.701164
3 16 2020-01-01 02:00:00 17.599297
4 16 2020-01-01 03:00:00 17.448940
5 16 2020-01-01 04:00:00 17.369642
6 16 2020-01-01 05:00:00 17.177364
7 16 2020-01-01 06:00:00 16.986202
8 16 2020-01-01 07:00:00 16.813074
9 16 2020-01-01 08:00:00 16.708633
10 16 2020-01-01 09:00:00 16.464092
11 16 2020-01-01 10:00:00 16.334290
12 16 2020-01-01 11:00:00 16.334015
13 16 2020-01-01 12:00:00 16.260014
14 16 2020-01-01 13:00:00 16.124367
15 16 2020-01-01 14:00:00 15.809680
16 16 2020-01-01 15:00:00 15.547085
17 16 2020-01-01 16:00:00 15.324635
18 16 2020-01-01 17:00:00 15.201200
19 16 2020-01-01 18:00:00 15.067950
20 16 2020-01-01 19:00:00 14.818552
21 16 2020-01-01 20:00:00 14.678573
22 16 2020-01-01 21:00:00 14.522839
23 16 2020-01-01 22:00:00 14.376727
In [70]:
nixtla_client.plot(tmp, timegpt_fcst_df,  max_insample_length=24*15)
Out[70]:
No description has been provided for this image

Historical forecast¶

In [71]:
timegpt_fcst_with_history_df = nixtla_client.forecast(
    df=tmp, h=1, freq='h', add_history=True,
)
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
INFO:nixtla.nixtla_client:Calling Historical Forecast Endpoint...
In [72]:
nixtla_client.plot(tmp, timegpt_fcst_with_history_df)
Out[72]:
No description has been provided for this image

All stations and backtesting¶

In [68]:
df = df[df.ds<'2019-08-15']
df
Out[68]:
unique_id ds y
0 9 2019-05-06 22:00:00 8.0
1 9 2019-05-06 23:00:00 8.0
2 9 2019-05-07 00:00:00 8.0
3 9 2019-05-07 01:00:00 8.0
4 9 2019-05-07 02:00:00 8.0
... ... ... ...
288235 128 2019-08-14 19:00:00 5.0
288236 128 2019-08-14 20:00:00 4.0
288237 128 2019-08-14 21:00:00 4.0
288238 128 2019-08-14 22:00:00 6.0
288239 128 2019-08-14 23:00:00 11.0

288240 rows × 3 columns

In [69]:
import os 
import numpy as np
from datetime import datetime
In [75]:
HORIZON = 12
path = f'predictions/timegpt/fcst_vienna_h{HORIZON}.csv'

if os.path.isfile(path):
    timegpt_fcst_df_backtest = pd.read_csv(path)
else: 
    timegpt_fcst_df_backtest = backtest_forecasting(df, datetime(2019,8,5), HORIZON)
    timegpt_fcst_df_backtest.to_csv(path, index=False)

rmse = compute_rmse(df, timegpt_fcst_df_backtest)
print(f'horizon={HORIZON}: rmse={rmse}')
horizon=12: rmse=4.838292942837482
  • TimeGPT horizon=1: rmse=2.3193235423384286
  • TimeGPT horizon=12: rmse=4.838292942837482
  • TimeGPT horizon=24: rmse=5.667080925461216
In [76]:
nixtla_client.plot(df, timegpt_fcst_df_backtest, max_insample_length=24*11)
Out[76]:
No description has been provided for this image

image.png

Results¶

Model Horizon RMSE
HistoricAverage 1 7.0229
HistoricAverage 12 7.0195
HistoricAverage 24 7.0426
SeasonalNaive 1 7.8703
SeasonalNaive 12 7.7317
SeasonalNaive 24 7.8703
AutoARIMA 1 2.2639
AutoARIMA 12 5.1505
AutoARIMA 24 6.3881
TimeGPT 1 2.3193
TimeGPT 12 4.8383
TimeGPT 24 5.6671

image-2.png

image.png

Remarks¶

Earlier paper by Zhang et al. (2016) "DNN-Based Prediction Model for Spatio-Temporal Data"¶

https://www.microsoft.com/en-us/research/wp-content/uploads/2016/09/DeepST-SIGSPATIAL2016_Zheng-2.pdf

Multi-step forcasting (i.e. longer forecasting horizon)¶

image-2.png image-3.png

More data is better ;-)¶

image.png

Potential error in RMSE reporting¶

The RMSE values of ARIMA and SARIMA for BikeNYC appear switched in Zhang et al. 2016 and 2017, respectively:

image.png

In [ ]: